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Seventeen different versions of a film designed to teach the assembly of tne 
breech block of the 40mm antiaircraft gun were tested on 2377 apprentice seamen in 
1948. Each version was shown to at least 100 men. in groups of about 30, in an 
effort to evaluate the relative effects on learning of certain content arid tempo 
characteristic s of specific ins It vciionai films. The film versions differed with respect 
to six variables, designated as Verbalization, Rate of Development. Nomenclature. 
Errors. *How it works, and Repetition. None of the men had ever assembled the 
breech block or similar mechanisms. They were grouped according to age. length of 
education. General Classification and Mechanical Aptitude Test Scores. Immediately 
after the film showing, each man was required to attempt to assemble the parts of 
the dismantled breech block Ten trials were given, and a time score obtained for 
each man. The most significant results were That a slow rate of development, the 
pointing out of errors to be avoided, and repetition of the assembly demonstration, 
markedly and positively contributed to the effectiveness of the film. (GO) 
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SUMMARY 



The purpose of this study was to evaluate the relative effects on 
learning of certain content and tempo characteristics of instructional films 
designed to teach an assembly task. Seventeen different versions of a f Un- 
designed to teach the assembly of the breech block of the 40mm. antiaircraft gun 
were produced for this study. These film versions differed with respect to 
six characteristics or variables, designated as Verbalization, Rate of 
Development, Nomenclature, Errors, "How-it-works", and Repetition. 

The film versions were produced by The Pennsylvania State College 
Motion Picture and Recording Studio during the spring of 1948, and were tested 
on apprentice seamen at the Great Lakes Naval Training Station that summer. 

Each version was shown to at least 100 men, in groups of approximately 30. 

In all, 2377 men participated as subjects. The film groups were equated with 
respect to age, length of education, General Classification Test score, and 
Mechanical Aptitude Test score. The men were all newly inducted, and none 
of them had had previous training in assembling the breech block or similar 
mechanisms. Immediately after the film showing, each man was required 
to attempt to assemble the parts of a disassembled breech block. Ten trials 
were given and a time score was obtained for each man. 

The most significant results were that a slow rate of development, 
the pointing out of errors to be avoided, and repetition of the assembly demon- 
stration contributed markedly and positively to the effectiveness of the film. 

The use or omission of the technical nomenclature, or the use of a high or 
low level of verbalization did not make significant differences in the effectiveness 
of otherwise comparable film versions. Presentation of the particular "how-it- 
works" sequence used in the experiment reduced, on the whole, the effectiveness 
of those versions in which it was included. 

The positive effects of a slow rate of development, the pointing out of 
errors, and repetition persisted in diminished degree but more or less con- 
sistently throughout the ten trials. 

The effects of the poor films were most persistent with respect to 
the poorer learners. 

Films which made use of slow rate of development, showing of common 
errors to be avoided, and repetition, were able, by themselves, to teach assembly 
of the breech block with a high degree of effectiveness. 



EFFECTS ON TRAINING OF EX PERIMENTAL FILM VARIABLES 
STUDY I- VFWRAT.T7.ATION, RATE OF DEVE LOPMENT , NOMENCLATURE 
ERRORST^HOW-IT- WORKS ", REPETITION I 

Nathan Jaspen 

INTRODUCTION 

Until recently there has been an almost complete lack of research 
that studied the characteristics of films themselves. The question of how much 
"better" or "worse" instructional films are than some other medium had been 
dealt with often. The question of what are the necessary characteristics of 
a "good" or "better" or "best" teaching film, however, had been posed only' 
rarely- 



Since August of 1947, the Instructional Film Research Program has 
taken some significant steps in this direction, beverai studies have attacked 
problems of the characteristics of the film itself. Roshal (3) dealt with the 
relative effectiveness of several ways of handling the visual material or pic- 
tures, and Zuckerman (5) dealt with characteristics of the commentary or 
sound track. These and other studies showed the effectiveness of instructional 
films is in large measure a function of the film techniques employed. The 
studies showed clearly that some film variables yielded more learning than others. 

STATEMENT OF THE PROBLEM 

As an early step in this kind of research, the present study was 
undertaken to investigate the relative effects on learning of certain character- 
istics or variables in fikns designed to teach an assembly skill. A basic pre- 
liminftry question to be answered was: What characteristics should training 
films have so that, by themselves and without help from instructors or books, 
they may be used to teach effectively in an emergency situation? 

The particular skill to be taught was assembly of the breech block 
of the 40mm antiaircraft gun. 

The particular variables whose effects on this learning were to be 
studied were Verbalisation, Rate of De velopment , Nomenclature , Errors, 

"How -it- works", and Repetition. These were defined as follows: 

1. Level of Verbalisation is the amount of narration used to describe 
the action of a film (expressed in average number of words per minute of film). 



1 This report is based on a dissertation by Jaspen^ 1), completed in May, 1949. 



Where the action of a film is the simple demonstration of an assembly task, 

Level of Verbalization is a relationship between how much is said and how much 
is demonstrated or done pictorially. If each frame (or each minute) of a film 
Is part of the demonstration, then how much is demonstrated can be expressed 
in minutes, and Level of Verbalization can be expressed as average number of 
words per minute of film. 

2. Rate of Development is the speed or tempo with which a given 

amount of material is covered pictorially. If one picture used four hundied 
feet of film to demonstrate how to assemble a breech block and a second 
picture used eight hundred feet of film to demonstrate how to assemble a 
breech block, then the first film had a faster rate of development than the 
second (i. e. , it took half as much time). The increase in film footage x (or 
time) may be the result of (1) taking more shots or (2) letting the camera 
rumjcnger for each shot or (3) a combination of more and longer shots, in j 

manipulating this variable in the preaeni experiment, sound tracks were varied 
only as much as necessary to correspond with the pictorial sequences. 

3. Nomenclature is the use of the technical names of the parts of 
the breech block. 

4. Errors is the showing of common errors to be avoided in perform- 
ing the assembly task. 

5. "How -it -works" is the use of a film sequence showing the principles 
of operation and function of the breech block. 

6. Repetition is the number of times the basic demonstration of the 
assembly task is presented in a film. 

With verbalization and rate of development, degrees were established 
for comparative investigation: high and low verbalization, slow and fast rates of 
development. In the case of nomenclature, errors, and "how-it-works", the 
presence or absence of these variables in the film were to be compared. (A 
film has, or does not have, the sequence showing errors to be avoided; a film 
has, or does not have, the technical names of the parts; a film has or does not 
have a sequence showing how the device works. ) With regard to repetition, a 
film had one, two, three, or four demonstrations of breech block assembly. 

To provide for an evaluation of the effectiveness of these variables, 
seventeen versiens of a film designed to teach assembly of the breech block were 
produced. A film version contained a planned pattern of variables. These version 
differed systematically from each other with regard to one or more of the six 
variables. The problem then was to determine which degree of each variable was 
the more effective, and which, among the six variables, were the more significant 
in making an instructional film of this type more effective. 
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EXPERIMENTAL DESIGN AND PROCEDURES 



The Task 



The task selected for this investigation had to meet certain requirements 



1 . Correct method of performance of the task should not be obvious 
on inspection, but should depend on definite instruction. 

2. The task should be unfamiliar to most persons. 

3. It druid .be neither too dil&rult nor too easy for the experimental 
population. 

4. An average individual, after instruction, should be able to 
achieve competent performance within a relatively short period. 

5. An objective criterion of successful performance of the task 
should be available. 



Assembly of the breech block of the 40mm. antiaircraft gun was 
selected as a task appropriately meeting all these requirements. Preliminary 
try-outs with men at The Pennsylvania State College confirmed the suitability 

of this task. 

Three meaningful and objective criteria of successful learning of 
the breech block assembly were selected for use in this study. 



The first criterion was successful assembly, regardless of speed. 
(In actuality, of course, it was necessary to impose a reasonable time limit. 



) 



The second criterion was length of time in seconds required for a 
successful assembly. 



The third criterion was a speed score . This score is the reciprocal 
of the time score (multiplied by 1000 to eliminate the decimal). An individual 
who failed to assemble the breech block in the time allowed got a speg d sco ^e 
of zero; the time score for such an individual was indeterminate (l. e. , it might 
be two hours"6rtwo years). Since means and standard deviations of time sco r e 
distributions could not be computed when failures to assemble the block were 
included, the means and standard deviations of speed scores could supply 
critical data that otherwise would have been unavailable. 
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The Film Variables 



Six variables were to be subjected to analysis in this study. These 
six variables and the design for studying them determined the nature of seven 
teen film versions which were used in the investigation. Two conditions or 
jl "levels" for each variable (except Repetition) were defined for study. These 

are described below: 

Verbalization. According to the average number of words per 
minute, a film was designated as having a high or low level of Verbalization. 
"High 1 * and "Low" were relative terms: of two films alike in every other 
respect, the one that had more words had a high level o f verbalization: the 
| one with fewer words had the low level of verbalization. 

m 

I It Should be und*»T«+ond that Verbalization was not expressed as the 

total number of words: it was expressed as the number of words per minute 

of film. 



It should be further understood that Verbalization in this study did 
not refer to the rate of speaking as such. The narrator spoke at a normal rate 
of * speed for all the film versions. The differences in amount of commentary 
from one film to another were controlled by varying the intervening periods of 

silence. 

jj Rate of Development. According to the speed with which a given amount 

of materiaTTEeaching the breech block assembly) was covered pictorially , a film 
was designated as having a slow or fast rate of development. In practice, a 
fast rate of development required that the film present visually (1) only the 
essential aspects of the action and (2) only for as long a time as necessary to 
give a reasonably complete presentation of the action. A slow rate of develop- 
[ ment required that the visuals or pictures give more time to each shot, use 

additional shots, and generally give a more extended pictorial coverage to the 

actions to be learned. 

Nomenclature. There are seven parts of the breech block; the firing 
pin, firing spring, firing spring cover, sear, sear spring, inner cocking lever, 
and outer cocking lever. These seven names were used in the commentary o 
some films. Tn other films, common names such as "part" or "piece" were 
used instead. The disassembled parts of the breech block and a special wrench 

are illustrated in Figure 1. 
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Errors. Each film included or omitted a sequence showing common 
errors to be avoided. In film versions which included showing of errors, the 
wrong method of performing a part of the assembly was, in each instance, 
inserted immediately after the demonstration^ the right method, and was 
clearly cautioned against as a wrong method. 

"How-it-works". Some film versions included a "How-it- works" 
sequence which showed the operation and "logic" of the breech block. Other 
films omitted this sequence. 



The hypothesis underlying the inclusion of the "how -it- works' 1 variable 
in this study was that the showing of meaningful relations between the elements of 
the task would facilitate the learning of the task (2, pp. 167-168J. The mo« 
meaningful the problem is to the learners, cne more readily it ic lcarr.e_ x -,. 

A question remains, however, whether the relations shown in the particular 
"how-it-works" sequence used in this study were not more appropriate to 
learning the breech block as an operating mechanism in a gun rather than to 
learning the breech block as an assembly task. Conclusions concerning this 
variable therefore should probably be limited and related to the specific sequence 

used in this study. 



Repetition. Repetition for the present experiment is defined to mean 
repetition of the demonstration of the assembly and not repetion in practic e of 
the assembly. Repetition, as defined, was achieved by demonstrating the assembly 
0 f +v>e breech block more than once in some film versions. In addition, one film 
version was produced which also repeated the "how-it-works" sequence and 
Errors as well as repeating the demonstration of assembly. 



The Film Versions 



In order to study the six variables, seventeen versions of the film 
were produced. In these seventeen versions, the six variables were systematically 
varied to provide a matrix of comparisons. 

In order to understand the degrees or levels of each variable and the 
combination of variables in each film version, it is convenient to assign symbols 
to each variable and to designate two "levels". Each variable is identifie 
therefore, by a letter: This letter as a capital designates one level of the 
variable; this letter in lower case designates the other level of the variable: 



) 
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Verbalization 
V - high 
v - low 



Errors 

E - shown 
e - not shown 



Rate of Development 
D - slow (long film) 
d - fast (short film) 



"How-it- works" 
H- shown 
h - not shown 



Nomenc lature 
N - used 
n - not used 



Repetition 
1, 2, 3, or 
4 sequences 



The seventeen film versions were identified by the letters of the 
alphabet, A throu R h Q. Using the symbols outlined above, the pattern of 
variables in each film version is presented in Table 1. 



In general, Films A and K served as the control versions for the 
experiment. 

General Outline of the Films . The experimental films were made up 
of the following sequences in the order indicated: 

I. Introduction. The anti-aircraft gun is shown in action. The 
picture dissolves to a closte-up of the breech block -- "the heart of the gun". 
The narrator says: "This film will show you how to assemble the breech 
block. Then you will be asked to do the job yourself. " This sequence begins 
all of the films. The footage on the gun was taken from Navy stock shots. 



2. "How-it-works" s equence ( when used) . The block is shown 
disassembled. "The principles of operation and the "logic" of the block are 
shown. In addition, there is an animation sequence showing a "transparent 
breech block in which the parts are visible. This sequence came from a 

military film. 

3. Assembly I sequence . This assembly sequence is present in all 
the film versions. The parts of the block are assembled in order. Errors ar 
shown in the sequence when the Errors variable is to be included in the film. 
Otherwise, Errors are not present, and the assembly proceeds straightforward 

4. Assembly II sequence. This assembly sequence is shorter than 
Assembly I. It is present" in all film versions except O. It does not include 
showing of Errors. 
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TABLE 1 



ID 



I 




\ 



\ 



STRUCTURE OF THE VERSIONS IN TERMS OF THE FILM VARIABLES 



Version 


Verbali- 

zation 


Rate of 

Develop- Nomen - 
ment clature 


Errors 


How- Repeti 

it Works tion 


Running 
- Time 
(Minutes) 


A 


V 


D 


N 


E 


H 


2 


18.0 


B 


V 


D 


N 


E 


H 


2 


18.0 


C 


V 


d 


N 


E 


H 


2 


11.0 


D 


V 


D 


n 


E 


H 


2 


18.0 


E 


V 


D 


N 


e 


H 


2 


15.2 


F 


V 


D 


N 


E 


h 


2 


13.8 


* G 


V 


D 


N 


E 


h 


3 


15.3 


H 


V 


D 


N 


e 


h 


2 


11.0 


I 


V 


d 


n 


E 

J 


H 


2 


11.0 


♦ J 


V 


d 


N 


e 


h 


2 


4.7 


K 


V 


d 


n 


e 


h 


2 


4.7 


L 


V 


D 


N 


E 


H 


4 


26.4 


M 


V 


d 


n 


E 


h 


2 


6.2 


N 


V 


d 


n 


e 


H 


2 


9.5 


O 


V 


d 


n 


e 


H 


1 


7.3 


P 


V 


D 


n 


e 


h 


2 


10.4 


Q 


V 


D 


n 


e 


h 


2 


10.4 


Variable 

Symbol 


Verbalization 


Rate of 
Deve lopment 


Nomenclature 


Errors How-it-Works 


Upper Case High 




Slow 


Used 




Shown 


Present 


Lower Case Low 




Fast 


Not Used 


Not. Shown 


Not Present 
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5. Repetition. Most of the film versions included two assembly 
sequences (assembly 1 and assembly II). Three versions (O, L, G) deviated 
from the general pattern with regard to repetition. Version O omitted 
Assembly II. In version L, several sequences were added after Assembly II 
and before the Conclusion. These consisted of a second "How-it- works" 
sequence (without animation), a second Assembly I sequence (with Errors), 
and a variation of Assembly II which closely resembles the latter. All of 
these repetition sequences proceed about twice as rapidly as the corresponding 
sequences in the first part of the film. Version G included the Assembly II 
variation, placed before the Conclusion, in addition to Assembly I and II. 

6. Conclusion. The parts are shown disassembled once more. The 
narrator says: "There are seven parts that fit into the breech block. Can you 
assemble them correctly?" This sequence was present in all films. 

Comparisons between versions for each variable. In order to 
establish the relative effectiveness of the different levels of each variable, 
the experimental design called for certain primary comparisons between 
film versions. 

Verbalization: 



Primary comparisons of the effectiveness of high and low Verbalization 
were made between versions A and B, in one case, and versions Q and P in the 
second case. Version A used 146 words per minute; version B used 83 words 
per minute. Version Q used 137 words per minute; ve rsion P used 73 words 
per minute. These figures represent average words per minute of film; they 
are arrived at by dividing the total number of words by the minutes of running 
time. 

Rate of Development: 

Primary comparisons of the effectiveness of slow and fast rates of 
development were between versions B and C, in the one case, and versions 
P and K, in the second case. Version B (slow rate of development) ran 
18.0 minutes; version C (fast development) ran 11.0 minutes. Version P 
(slow rate of development) ran 10. 4 minutes; version K (fast rate of develop- 
ment) ran 4.7 minutes. (Versions B and C were longer than versions P and 
K, because they included "H>w- it- works", and Errors. ) 
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Nomenc lature : 



Primary comparison of the effectiveness of the presence and absence 
of Nomenclature was made between version D and the positive control version A. 
in the one case, and between version J and the negative control version K. in the 
second case. D differs from A, and J differs from K. only with respect to this 

variable. . 

Errors: 

Comparison of the effectiveness of the presence and absence of Errors 
was made between four pairs of films which differed only in so far as they showed, 
or did not show. Errors to be avoided. These four comparisons were between 
versions A and E, F and H, I and N, and M and K. Versions A, F , I, and M all 
included Errors . Versions E, H, N, and K all omitted Errors . 

"How -it- Works" : 

Comparisons of the effectiveness of the presence and absence of 
"How-it- Works" were made between versions A and F, E and H, I and M, 
and N and-K. The first version of each of these pairs (A, E, I, and N) has 
a ,r How- it- Works" sequence; the second version in each pair (F, H, M, and K) 
does not have a "How-it- Works" sequence. 



Repetition : 

Comparison of the effectiveness of two assembly demonstrations and 
three assembly demonstrations was made between version F (two demonstrations) 
and G (three demonstrations). The films were alike in all other respects. 

Comparison of the effectiveness of one assembly demonstration and 
two assembly demonstrations was made between versions O (one demonstration) 
ISd N (two demonstrations). The two films were alike in all other respects. 

A special comparison was made between version L. and version A. 
Version L used Assembly I and Assembly II. each shown twice (a total of four 
assembly demonstrations); version A used Assembly I and Assembly II. neither 
one repeated (a total of two demonstrations). In addition, version L also included 
a repetition of Errors (as part of the first assembly sequence, shown twice) and 
a repetition of the "How-it-Works" sequence. 
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The Test Population 



The men to whom the experimental film versions were shown were 
apprentice seamen at the Great Lakes Naval Training Station, Great Lakes, Illinois. 

Thirty men were selected from each of 81 companies, on the basis 
of age, education. Navy General Classification test score, and Navy Mechanical 
Aptitude test score. All of the men selected were aged between 16 years, 8 months, 
and 21 years, 8 months. They had at least six years of formal education; they. had 
Navy General Classification Test scores not lower than 40 and not higher than 60. 
They had Navy MA scores not lower than 40 and not higher than 60. These require- 
ments insured that the groups selected for training and testing would be com- 
parable and homogeneous. Homogeneity was further achieved by restricting 
the experimental population to those individuals whose combined General Class- 
ification Test and Mechanical Aptitude scores totalled between 90 and 110. 

Since 30 men were selected from each of 81 xompanies, a total of 
2430 might have qualified for the experiment. Data for 53 of these men were 
not used, however, because the men were absent, or had hand injuries, or 
were found, on review, not to meet the criteria of selection. No man was 
dropped because of his breech block performance score. 

The Proctors 

A detail of proctors was assigned to the experiment for each week of 
the study. Each detail consisted of 36 men. This included a proctor for each 
of the 30 assemblers being tested at a given time. Proctors were selected only 
from companies which did not provide assemblers; their "GCT" scores were 
for the most part superior (above 60). 

Each detail of proctors was given four hours of instruction in reading 
standardized instructions, operating a stop-watch, keeping records, scoring 
breech block assembly, and other necessary skills. 

Experimental Procedure 

A standard motivational speech was given to each group of thirty men. 

The thirty men were then shown one of the film versions. After the film showing 
the men were directed to the testing room, which was equipped with ten long 
tables. On the tables, for each man, was a Test Record Sheet, with the man’s 
name and service number and a number identifying the proctor assigned to test him. 
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When all the assemblers were in tlieir assigned positions, a signal 
was given for testing to start. Proctors r^ad a standard set of directions, laid 
out the parts of the disassembled breech block in a standardised pattern and 
told the assemblers to begin work. Au assembler was timed from the moment 
he touched the first part until he completed the assembly. The proctor recorded 
his time (in seconds) &r each of ten assembLy trials. If an assembler failed to 
assemble the block in ten minutes, it was disassembled and he was told to try 
again. If he succeeded the second time, he continued for a total of ten trials. 

If he failed, he was dismissed, and his performance was recorded as a failure. 

RESULTS 

The Relative Effectiveness of the Seventeen Film Versions 



While the main interest of this study is in the relative effectiveness 
of the film variables, it seems pertinent to inquire, first, whether the film 
versions resulted in significant learning, and if so, whether there were dif- 
ferences in the effectiveness of the different film versions. 

Table 2 establishes that learning does occur exclusively from film 
instruction. In the case of the most effective films as many as 98% of the 
subjects succeeded in learning to assemble the breech block, within a time 
limit of ten minutes. * 

I 

That the different films varied greatly in effectiveness is also 
established by Table 2. Less than half the subjects who saw the least 
effective version (O) were able to complete the assembly within the first 
ten minutes; the median time for this film group was 749 seconds. The 
median for the most effective film was 101 seconds. 

If we arrange the versions in order of effectiveness as measured 
by the criterion of mean speed, (Table 2), we find the following to be true; 

1. The most effective version (L) had a slow rate of development, 
showed errors, and included four assembly demonstrations. 

2. The second most effective version (G) had a slow rate of develop- 
ment, showed errors, and included three assembly demonstrations. 

3. The vnri four most effective versions (F, D, B, A) had a slow 
rate of development, showed errors, and included two assembly demonstrations. 

4. The following seven versions in order of effectiveness (I, H, P, Q, 
E, C, M) each included either a slow rate of development or errors, but not both. 
Each included two assembly demonstrations. 



In Study II it was found that of a control group which received no instruction 
whatever, only 15% succeeded in assembling the breech block in a ten minute 
trial. 
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TABLE 2 



RELATIVE EFFECTIVENESS OF THE FILM VERSIONS IN 
TERMS OF SEVERAL CRITERIA 



Film 


N 


Mean speed 
score on 
first attempt 


Median 
time score 
on first 
attempt 
(in seconds) 


Percent 
successful on 
first attempt 
600 second 
time limit 


Percent 
successful on 
first attempt 
1200 second 
time limit 


A 


233 


7.5 


150 


88.0 


92.7 


B 


118 


7.7 


134 


87.3 


99.2 


C 


116 


6.5 


174 


88.8 


97.4 


D 


147 


8.2 


127 


88. 4 


93.2 


E 


141 


7. 1 


157 


87.9 


94.3 


F 


118 


9.3 


106 


98. 3 


99.2 


G 


119 


99 


103 


95.8 


97.5 


H 


116 


7.4 


162 


89.7 


93. 1 


I 


116 


7.5 


138 


92.2 


97.4 


J 


150 


4.8 


332 


64.7 


83.3 


K 


266 


4.6 


346 


62.4 


75.6 


L 


116 


10.5 


101 


98.3 


99. 1 


M 


117 


6.5 


164 


91.5 


99. 1 


N 


150 


5.6 


289 


74.7 


88.0 


O 


118 


2. 1 


749 


45.8 


695 


P 


116 


7.3 


156 


91.4 


97.4 


Q 


120 


7.2 


152 


90.0 


97.5 


Range of 
Standard 
Errors - 


. 2 to . 5 


• • 


1 . 2 to 4. 6 


. 8 to 4. 2 



o 
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5. The 1 allowing three versions in order of effectiveness (N, J, K) 
had a rapid rate of development, did not show errors, and had two assembly 
demonstrations . 

6. The least effective version (O) had a rapid rate of development, 
did not show errors, and included only one assembly demonstration. 

It appears from this tabulation that the most potent variables are a 
slow rate of development, the showing of errors to be avoided, and repetition. 
This conclusion is substantiated by the statistical analysis (see Table 4 and 5). 

Effects of the Versions on Subsequent Trials. 



The analysis for the average learners gives us no assurance that 
be low-average and above-average learners will be affected by films in the 
same way. If we compare speed scores for the poorest and best assemblers 
(lower and upper quartiles) from one version to another, do we obtain the same 
results that we do for median or average learners? 

Table 3 shows the speed scores on the tenth trial at the .upper quartile; 
median, and lower quartile, for the seventeen film versions. The range of 
medians is from 31 to 42, On the first trial the range of median speed scores 
was from 1 to 10. It would appear from this that while the differential effects 
of the films do diminish, on a relative basis, they nevertheless persist in 
some degree even until the tenth trial. Approximately the same range is 
found in the distribution of upper quartiles in the tenth trial. 

On the other hand, the lower quartile scores diverge much more 
widely from each other. Figure 2 illustrates the learning curves for versions 
A and K at the median, the upper quartile, and the lower quartile. The greater 
divergence at the lower quartile is well illustrated in this graph. 

It would, therefore, appear that the least effective films persist 
in their negative effects, especially on the poorer learners. 

Relative Effectiveness of the Levels of each Film Variable. 



To determine the effectiveness of the levels of each variable, 
comparisons were made between pairs of versions which differed from each 
other only with respect to this variable. These comparisons were based on 
mean speed scores on the first successful assembly. 
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TABLE 3 



SPEED SCORES ON TRIAL 10 FOR THE SEVENTEEN FILM 
VERSIONS: UPPER QUARTILES, MEDIANS, AND LOWER 

QUARTILES 

Speed Score* (1000 divided by time in seconds) 





Upper 
Quart! le 


Median 


Lower 

Quartile 


A 


43 


37 


29 


B 


43 


36 


29 


C 


45 


38 


27 


D 


48 


38 


29 


E 


48 


38 


32 


F 


50 


40 


32 


G 


50 


42 


33 


H 


45 


37 


26 


I 


45 


38 


30 


J 


43 


33 


24 


K 


43 


33 


14* 


L 


48 


40 


31 


M 


48 


38 


29 


N 


45 


37 


27 


O 


38 


31 


14 


P 


43 


34 


27 


Q 


45 


37 


29 



* Score obtained by extrapolation from earlier trials. 
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Fig. 2. Learning Curves for Films A and K for Ten Trials: 
Upper Quartile, Median, and Lower Quartile. 
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The mean speed score differences between pairs of films alike 
except for a single variable are summarized in Table 4. 

The results of these comparisons and their implications for^Cach 
variable are discussed below: 

Verbalization. Version A was compared with version B, and version 
Q was compared with version P. In both comparisons, differences in effective- 
ness between high and low verbalization were not significant. 

Rate of Development. Version B »s compared with version C, and 
version P ^Tc^npared with version K. In both comparisons, the versions 
with slow development (B and P) were superior to those with fast development. 

The superiority of P over K was significant to the 0. 1% level of confidence. 

Nomenclature. Version A, which used technical nomenclature, was 
inferior to version D which lacks technical nomenclature. Version J, which 
used nomenclature, was slightly superior to version K, which lacks nomenclature 
Comparisons of H and Q, and I and C respectively also yield inconsistent results. 
No significant differences were established. 

Errors. Versions A and E were compared. Version A shows Errors 
version E does not. Version A is moderately superior to version E. 

Version M and K were compared. Version M shows Errors; version 
K does not. Version M is appreciably superior to version K and the difference 
is statistically significant at a high level. 

Similarly, version F was compared with version H, and version I 
was compared with version N. In each comparison, the version which showed 
Errors was superior to the version which lacked Errors . 

• four comparisons appear to establish the effectiveness of point- 

ing out (ih training filmic bommoiieerrors to bb. ;yvbided.an T aifeembly tasks. In 
three of the comparisons these differences are statistically significant. 

"How -it -Works”. Four comparisons are available for this variable. 
Results for two of these are positive; results for the other two are negative. 

None of the differences is significant. 
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TABLE 4 






DIFFERENCES AND SIGNIFICANCE OF DIFFERENCES BETWEEN 
CONTROL AND EXPERIMENTAL FILMS WITH REGARD TO EACH FILM VARIABLE 
CONSIDERED SINGLY* IN TERMS OF THE MEAN SPEED SCORES EARNED ON THE 

FIRST ASSEMBLY ATTEMPT 



Variable and 
Ver$ion Pairs 


M x 


m 2 


Diff. 1 

• 


o-Diff. 


Degrees of 
Freedom 2 


t- 

ratio 


Verbalization 














A and B 


7.50 


7.69 


19 


.86 


10 


. 22 


Q and P 


7. 19 


7.27 


-.08 


.49 


6 


. 16 



Rate of Development 





B and C 


7.69 


6.48 


1.21 


. 55 


6 


2.20 


P and K 


7.27 


4.60 


2.67 


.56 


11 


4.77** 


Nomenclature 


A and *D 


7.50 


8. 21 


-. 71 


.93 


11 


.76 


J and K 


4,78 


4.60 


. 18 


.62 


12 


.29 


H and Q 


7.36 


7. 19 


. 17 


.74 


6 


.23 


C and I 


6. 48 


7.48 


-1.00 


.49 


6 


2.04 


Errors 


A 1 * and E 


7.50 


7. 12 


. 38 


.96 


11 


.40 


M and K 


6.48 


4.60 


1.88 


.59 


11 


3. 19** 


F and H 


9.29 


7.36 


1.93 


.61 


6 


3. 16* 


I and N 


7.48 


5.57 


1.91 


.78 


7 


2.45* 


"How-it- Works" 


A and F 


7.50 


9.29 


-1.79 


.81 


10 


2.21 


N and K 


5.57 


4.60 


.97 


.66 


12 


1.47 


E and H 


7. 12 


7. 36 


-.24 


1.08 


7 


.22 


I and M 


7,48 


6.48 


1.00 


: 53 


6 


1.89 


Repetition 


L and A 


10.50 


7. 50 


3.00 


.92 


10 


3. 26** 


G and F 


9.90 


9.29 


.61 


.45 


6 


1.36 


N and O 


5.57 


2.09 


3.48 


. 72 


7 


4.83** 



* Significant at the 5 percent level 

** Significant at the 1 percent level 

1 Sign of difference is from "positive" version minus "negative" version. 

2 Deg rees of freedom are based on the number of subgroups (details) that saw 
each version. 
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